The present invention relates to the field of data analysis.
Decision making is an important process people go through daily. Some decisions are made unconsciously, some are based on instinct, but in most cases, the evidence-based decision making is considered the best way. Examples may come from various domains such as politics, finance, healthcare etc. However, to make evidence-based decisions, people may require good and reliable evidence in real-time. As the quantity of available data grows, gathering the relevant evidence becomes a challenging resource staining task.
Text mining, also referred to as text analytics (or analysis), is often defined as the automated process of deriving high-quality information from text (specifically, large amounts of text) via computer processing. High-quality information is typically derived through the devising of patterns and trends through means such as statistical pattern learning and machine learning. Text mining usually involves the process of structuring the input text (usually parsing, along with the addition of some derived linguistic features and the removal of others, and subsequent insertion into a database), deriving patterns within the structured data, and finally evaluation and interpretation of the output. ‘High quality’ in text mining usually refers to some combination of relevance, novelty, and interestingness. Typical text mining tasks may include text categorization, text clustering, concept/entity extraction, production of granular taxonomies, sentiment analysis, document summarization, and entity relation modeling (i.e., learning relations between named entities).
Text analysis may involve information retrieval, lexical analysis to study word frequency distributions, pattern recognition, tagging/annotation, information extraction, data mining techniques including link and association analysis, visualization, and predictive analytics. The overarching goal may be, essentially, to turn text into data for analysis, via application of methods such as natural language processing (NLP) and analytical methods.
With continuous advancements and an increase in user popularity, data mining and text analysis technologies may serve as an invaluable resource across a wide range of disciplines.
The technology is now broadly applied for a wide variety of needs, including government, research and business needs. Applications of text analysis may include intelligence, security, e-discovery, records management, publishing, automated ad placement, social media monitoring, scientific discovery etc.
The foregoing examples of the related art and limitations related therewith are intended to be illustrative and not exclusive. Other limitations of the related art will become apparent to those of skill in the art upon a reading of the specification and a study of the figures.